[0.9.1][Feature]Moe alltoallv communication optimization for unquantized RL training sence & alltoallv support dpo #1547
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 12 commits
July 1, 2025 09:51
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
added 3 commits
July 1, 2025 14:03
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
…training sence & alltoallv support dpo Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: duyangkai <duyangkai@huawei.com>
| # at different points based on MoE settings as late as possible. | ||
| # Valid sync points are "before_permutation_1", "before_ep_alltoall", | ||
| # "before_finish", and "no_sync". | ||
| self.cuda_sync_point = "no_sync" |
There was a problem hiding this comment.
Why use this naming, seems little bit unsuitable in vllm-ascend
There was a problem hiding this comment.
cuda_sync_point is already renamed to device_sync_point
Signed-off-by: duyangkai <duyangkai@huawei.com>
… into v0.9.1-dev # Conflicts: # tests/ut/test_distributed_tensor_parallel.py # tests/ut/test_moe_util.py # tests/ut/test_token_dispatcher.py # vllm_ascend/ascend_forward_context.py # vllm_ascend/envs.py # vllm_ascend/models/moe_block.py # vllm_ascend/models/qwen3_dbo.py # vllm_ascend/ops/fused_moe.py # vllm_ascend/ops/moe_dispatcher/token_dispatcher.py
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
added 11 commits
July 12, 2025 23:43
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
added 2 commits
July 12, 2025 23:44
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: weijinqian_v1 <weijinqian@huawei.com>
Signed-off-by: duyangkai <duyangkai@huawei.com>
ganyi1996ppo
approved these changes
Jul 14, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[Feature]Moe alltoallv communication optimization for unquantized RL training sence & alltoallv support dpo
Introduction
This PR introduces two key optimizations for MoE model performance:
Efficient Token Dispatcher:
alltoallv_seqtoken dispatcher (adopted from NVIDIA Megatron and Ascend MindSpeed)VLLM_ASCEND_ENABLE_MOE_ALL2ALL_SEQ=1DBO Support for alltoallv_seq:
alltoallv_seqdispatcher to support DBO (Dual Batch Overlap)VLLM_ASCEND_ENABLE_MOE_ALL2ALL_SEQ=1VLLM_ASCEND_ENABLE_DBO=1Performance Improvements
Testing on Qwen3-30B-A3B shows nearly 2x throughput improvement compared to the original alltoall implementation.